MACHINE READING COMPREHENSION

Have my arguments been replied to? Argument Pair Extraction as Machine Reading Comprehension (2022.acl-short)

Challenge:	Experimental results show that unified model outperforms other models that treat encoding and matching separately.
Approach:	They evaluate a unified model with Transformer layers for machine reading comprehension . they find that the model learns different modeling strategies compared with previous models .
Outcome:	The unified model outperforms models with Transformer layers on the machine reading comprehension task.

Copied to clipboard

Jianzhu Bao, Jingyi Sun, Qinglin Zhu, Ruifeng Xu

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data (2021.findings-emnlp)

Challenge:	Existing studies identify argument pairs indirectly by predicting sentence-level relations between two documents, neglecting the holistic argument-level interactions.
Approach:	They propose to use machine reading comprehension to extract argument pairs from two documents . they propose to employ an AM query to identify all arguments in two documents, then an APE query to extract its paired arguments from another document.
Outcome:	The proposed method outperforms the state-of-the-art method by 7.11% in F1 score.

Copied to clipboard

Dian Yu, Kai Sun, Dong Yu, Claire Cardie

Cross-Task Knowledge Transfer for Query-Based Text Summarization (D19-58)

Challenge:	despite considerable progress, most machine reading comprehension tasks lack sufficient training data to fully exploit powerful deep neural network models.
Approach:	They propose to use QA data to generate more training data for machine reading comprehension tasks by crowdsourcing . they first collect a large-scale multiple-choice QA dataset for Chinese, ExamQA, and then use incomplete, yet relevant snippets returned by a web search engine as the context for each QA instance.
Outcome:	The proposed model improves a Chinese MRC task with +5.1% accuracy and +3.8% exact match.

Copied to clipboard

Elozino Egonmwan, Vittorio Castelli, Md Arafat Sultan

Text Generation

Challenge:	Existing methods for summarization data corpora are limited to extractive and abstractive summarizing.
Approach:	They propose to use machine reading comprehension (MRC) and query-based text summarization to produce extractive and abstractive summaries from pre-trained MRC and MT models.
Outcome:	The proposed model outperforms existing methods on CNN/Daily Mail and Debatepedia datasets and can be used as a baseline for future systems.

Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)

Copied to clipboard

Xiepeng Li, Zhexi Zhang, Wei Zhu, Zheng Li, Yuan Ni, Peng Gao, Junchi Yan, Guotong Xie

Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021.starsem-1)

Challenge:	Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations.
Approach:	They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention.
Outcome:	The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets.

Copied to clipboard

Ziqing Yang, Yiming Cui, Chenglei Si, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

On the Robustness of Reading Comprehension Models to Entity Renaming (2022.naacl-main)

Challenge:	Neural networks are vulnerable to adversarial examples that have been mixed with certain perturbations.
Approach:	They propose a novel adversarial training method that perturbs the embedding matrix instead of word vectors to differentiate the roles of passages and questions.
Outcome:	The proposed method is effective universally and further improves the performance of MRC tasks.

Copied to clipboard

Jun Yan, Yang Xiao, Sagnik Mukherjee, Bill Yuchen Lin, Robin Jia, Xiang Ren

Dice Loss for Data-imbalanced NLP Tasks (2020.acl-main)

Challenge:	SpanBERT model is more robust than RoBERTa, despite having similar accuracy on unperturbed test data.
Approach:	They propose a pipeline to replace entity names with names from a variety of sources.
Outcome:	The proposed model performs worse when entities are renamed, the authors show . SpanBERT, which is pretrained with span-level masking, is more robust than RoBERTa .

Copied to clipboard

Xiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang, Fei Wu, Jiwei Li

Syntactic Text Processing

Challenge:	Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension .
Approach:	They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks.
Outcome:	The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks.

MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension (2020.aacl-main)

Copied to clipboard

Huaishao Luo, Yu Shi, Ming Gong, Linjun Shou, Tianrui Li

Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension (2022.acl-long)

Challenge:	Existing methods to predict the start and end positions of answer spans generate two probability vectors.
Approach:	They propose a method that extends the probability vector to a probability matrix.
Outcome:	The proposed method improves on SQuAD 1.1 and three other question answering benchmarks.

Copied to clipboard

Linjuan Wu, Shaojuan Wu, Xiaowang Zhang, Deyi Xiong, Shizhan Chen, Zhiqiang Zhuang, Zhiyong Feng

VlogQA: Task, Dataset, and Baseline Models for Vietnamese Spoken-Based Machine Reading Comprehension (2024.eacl-long)

Challenge:	Existing methods to zero-shot transfer knowledge from rich-resource to low-resourced languages are limited due to linguistic discrepancies in different languages.
Approach:	They propose a multilingual MRC framework equipped with a Siamese Semantic Disentanglement Model to disassociate semantics from syntax in models learned by multilingual pre-trained models.
Outcome:	The proposed model disassociates semantics from syntax in multilingual models.

Copied to clipboard

Thinh Ngo, Khoa Dang, Son Luu, Kiet Nguyen, Ngan Nguyen

Why Machine Reading Comprehension Models Learn Shortcuts? (2021.findings-acl)

Challenge:	Existing datasets for machine reading comprehension tasks in Vietnamese focus on written documents, such as Wikipedia articles, online newspapers, or textbooks.
Approach:	They propose to capture Vietnamese spoken language in natural settings and use it to create a machine-learning corpus for machine reading comprehension tasks.
Outcome:	The proposed corpus consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube .

Copied to clipboard

Yuxuan Lai, Chen Zhang, Yansong Feng, Quzhe Huang, Dongyan Zhao

Frustratingly Simple Few-Shot Slot Tagging (2021.findings-acl)

Challenge:	Existing studies show that many MRC models learn shortcuts to outwit benchmarks, but the performance is unsatisfactory in real-world applications.
Approach:	They propose to use shortcut questions to analyze learning difficulty of MRC models . they propose to analyze the learning difficulty regarding shortcut and challenging questions .
Outcome:	The proposed methods show that a large proportion of shortcut questions in training data make models rely on shortcut tricks excessively.

Copied to clipboard

Jianqiang Ma, Zeyu Yan, Chang Li, Yang Zhang

GCRC: A New Challenging MRC Dataset from Gaokao Chinese for Explainable Evaluation (2021.findings-acl)

Challenge:	Existing fewshot methods for slot tagging are weak in encoding slot name semantics and slot dependencies.
Approach:	They propose a simple and effective few-shot model for slot tagging which incorporates machine reading comprehension (MRC) using source domain and target domain data.
Outcome:	The proposed model outperforms state-of-the-art methods on the SNIPS dataset.

Copied to clipboard

Hongye Tan, Xiaoyue Wang, Yu Ji, Ru Li, Xiaoli Li, Zhiwei Hu, Yunxiao Zhao, Xiaoqi Han

Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021.acl-short)

Challenge:	Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities.
Approach:	They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC .
Outcome:	The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way.

Copied to clipboard

Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang

Leveraging Type Descriptions for Zero-shot Named Entity Recognition and Classification (2021.acl-long)

Challenge:	Recent work focuses on question answering based on machine reading comprehension . current approaches treat QA as extracting a consecutive piece of text to a given question.
Approach:	They propose a generative QA model that incorporates an extractive mechanism into a model.
Outcome:	The proposed model improves quality and semantic accuracy over baseline models.

Copied to clipboard

Rami Aly, Andreas Vlachos, Ryan McDonald

Entity-Relation Extraction as Multi-Turn Question Answering (P19-1)

Challenge:	Named entity recognition and classification (NERC) tasks require annotated data for the target classes during training.
Approach:	They propose a novel approach that leverages the fact that textual descriptions for many entity classes occur naturally.
Outcome:	The proposed approach outperforms baselines adapted from machine reading comprehension and zero-shot text classification.

Copied to clipboard

Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, Jiwei Li

Learning to Generate Questions by Learning to Recover Answer-containing Sentences (2021.findings-acl)

Challenge:	Identifying entities and their relations is the prerequisite of extracting structured knowledge from unstructured raw texts.
Approach:	They propose a new paradigm for the task of entity-relation extraction . they cast the task as a multi-turn question answering problem .
Outcome:	The proposed paradigm significantly outperforms previous best models on the ACE and CoNLL04 datasets.

Copied to clipboard

Seohyun Back, Akhil Kedia, Sai Chetan Chinthakindi, Haejun Lee, Jaegul Choo

A Query-Parallel Machine Reading Comprehension Framework for Low-resource NER (2023.findings-emnlp)

Challenge:	Recent research has focused on synthetically generating a question from a given context and an annotated answer by training an additional generative model.
Approach:	They propose a method that learns to generate contextually rich questions by recovering answer-containing sentences.
Outcome:	The proposed approach improves the quality and accuracy of existing models and achieves comparable results to the state-of-the-art on MS MARCO and NewsQA.

Copied to clipboard

Yuhao Zhang, Yongliang Wang

Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling (2022.naacl-main)

Challenge:	Named entity recognition (NER) is a fundamental task in natural language processing.
Approach:	They propose a query-parallel MRC-based approach to named entity recognition . the model is trained with parameter-efficient tuning technique, making it more data-efficient .
Outcome:	The proposed model performs competitively against strong baseline methods in resource-rich settings and achieves state-of-the-art results in low-resource settings.

Copied to clipboard

Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Daxin Jiang

Teaching Machine Comprehension with Compositional Explanations (2020.findings-emnlp)

Challenge:	Existing methods to train cross-lingual pre-trained language models have shown great success in cross-linguistic sequence labeling tasks.
Approach:	They propose a cross-lingual language informative span masking task to eliminate the objective gap between pre-training and fine-tuning stages.
Outcome:	The proposed method surpasses the state-of-the-art methods on multiple benchmarks even with limited pre-training data.

Copied to clipboard

Qinyuan Ye, Xiao Huang, Elizabeth Boschee, Xiang Ren

Explainability & Interpretability in NLP

Challenge:	Recent advances in machine reading comprehension rely heavily on large-scale annotated corpora, which are timeconsuming and costly to collect.
Approach:	They propose to use semi-structured explanations to “teach” machines reading comprehension using a small number of semi-structural explanations that explicitly inform machines why answer spans are correct.
Outcome:	The proposed method achieves 70.14% F1 score with supervision from 26 explanations on the SQuAD dataset, comparable to plain supervised learning using 1,100 labeled instances yielding a 12x speed up.

To What Extent Do Natural Language Understanding Datasets Correlate to Logical Reasoning? A Method for Diagnosing Logical Reasoning. (2022.coling-1)

Copied to clipboard

Yitian Li, Jidong Tian, Wenqing Chen, Caoyun Fan, Hao He, Yaohui Jin

Subword-augmented Embedding for Cloze Reading Comprehension (C18-1)

Challenge:	Reasoning and knowledge-related skills are considered as fundamental skills for natural language understanding (NLU) tasks.
Approach:	They propose a method to diagnose correlations between an NLU dataset and a specific skill.
Outcome:	The proposed method is able to diagnose correlations between dataset and logical reasoning skill on 8 MRC and 3 NLI datasets.

Copied to clipboard

Zhuosheng Zhang, Yafang Huang, Hai Zhao

Representation Learning

Challenge:	Existing models for machine reading comprehension use word and character representations, but character is not the minimal unit.
Approach:	They propose to use subword rather than character for word embedding enhancement . they also empirically explore different augmentation strategies on subword-augmented embedded embedders .
Outcome:	The proposed model outperforms state-of-the-art models on public datasets.

Stochastic Answer Networks for Machine Reading Comprehension (P18-1)

Copied to clipboard

Xiaodong Liu, Yelong Shen, Kevin Duh, Jianfeng Gao

RoR: Read-over-Read for Long Document Machine Reading Comprehension (2021.findings-emnlp)

Challenge:	Several recent MRC models employ multi-step reasoning . we show that the use of a stochastic prediction dropout improves robustness .
Approach:	They propose a stochastic answer network that simulates multi-step reasoning in machine reading comprehension.
Outcome:	The proposed model improves robustness and results competitive with state-of-the-art models on the Stanford Question Answering Dataset and Microsoft MAchine Reading COmprehension Dataset.

Copied to clipboard

Jing Zhao, Junwei Bao, Yifan Wang, Yongwei Zhou, Youzheng Wu, Xiaodong He, Bowen Zhou

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information (2021.acl-long)

Challenge:	Existing models for machine reading comprehension are limited to individual chunks due to encoding length constraint.
Approach:	They propose a read-over-read method that expands the reading field from chunk to document by predicting regional answers for each chunk.
Outcome:	Extensive experiments on QuAC and TriviaQA show that the proposed model performs well for long document reading.

Copied to clipboard

Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu, Jiwei Li

From Multiple-Choice to Extractive QA: A Case Study for English and Arabic (2025.coling-main)

Challenge:	ChineseBERT model incorporates glyph and pinyin information of Chinese characters into pretraining . proposed model achieves new performance boost over baseline models with fewer training steps .
Approach:	They propose a ChineseBERT model that incorporates glyph and pinyin information into pretraining . the glyph embedding is obtained based on different fonts of a character, and the pinyink embeddment characterizes the pronunciation of Chinese characters.
Outcome:	The proposed model achieves new performance boosts over baseline models with fewer training steps.

Copied to clipboard

Teresa Lynn, Malik H. Altakrori, Samar M. Magdy, Rocktim Jyoti Das, Chenyang Lyu, Mohamed Nasr, Younes Samih, Kirill Chirkunov, Alham Fikri Aji, Preslav Nakov, Shantanu Godbole, Salim Roukos, Radu Florian, Nizar Habash

Dependency Parsing as MRC-based Span-Span Prediction (2022.acl-long)

Challenge:	Recent years have brought about very fast developments in Natural Language Processing (NLP), but many other languages are overlooked due to limited resources.
Approach:	They propose to repurpose a multilingual BELEBELE dataset for a task of extractive QA in the style of machine reading comprehension.
Outcome:	The proposed approach could be used to extract QA in the style of machine reading comprehension.

Copied to clipboard

Leilei Gan, Yuxian Meng, Kun Kuang, Xiaofei Sun, Chun Fan, Fei Wu, Jiwei Li

Syntactic Text Processing

Challenge:	Existing methods for dependency parsing address the issue that edges should be constructed at the text span/subtree level rather than word level.
Approach:	They propose a method that constructs dependency trees by directly modeling span-span relations by modeling subtree-subtree relationships.
Outcome:	The proposed method constructs dependency trees by modeling span-span relations . it can retrieve missing spans in the span proposal stage, which leads to higher recall .

Probing into the Root: A Dataset for Reason Extraction of Structural Events from Financial Documents (2021.eacl-main)

Copied to clipboard

Pei Chen, Kang Liu, Yubo Chen, Taifeng Wang, Jun Zhao

An MRC Framework for Semantic Role Labeling (2022.coling-1)

Challenge:	Existing methods for event reason extraction are far from resolving this problem.
Approach:	They propose a task to extract causal explanations from document-level texts . they use a dataset FinReason for evaluation to provide Reasons annotation for financial events .
Outcome:	The proposed task performs better than existing methods on a dataset of 8,794 documents, 12,861 financial events and 11,006 reason spans.

Copied to clipboard

Nan Wang, Jiwei Li, Yuxian Meng, Xiaofei Sun, Han Qiu, Ziyao Wang, Guoyin Wang, Jun He

Semantic Parsing

Challenge:	Existing work on semantic role labels ignores the semantic connection between the two tasks . et al. (2010) defined two types of semantic roles: core roles and non-core roles.
Approach:	They propose to use machine reading comprehension to bridge the gap between these two tasks . they formalize predicate disambiguation as multiple-choice machine reading understanding .
Outcome:	The proposed framework achieves state-of-the-art or comparable results to previous work . it uses the descriptions of candidate senses of a given predicate as options to select the correct sense .

Ask to Learn: A Study on Curiosity-driven Question Generation (2020.coling-main)

Copied to clipboard

Thomas Scialom, Jacopo Staiano

Interactive Machine Comprehension with Information Seeking Agents (2020.acl-main)

Challenge:	Existing work on Question Generation focuses on generating relevant questions given text with an answer . human ability to ask questions goes beyond evaluation of reading comprehension .
Approach:	They propose a novel text generation task based on a conversational question-asking dataset . they investigate automated metrics to measure different properties of Curious Questions .
Outcome:	The proposed task is based on a conversational Question Answering dataset . the results show that humans tend to ask questions with the goal of obtaining new information .

Copied to clipboard

Xingdi Yuan, Jie Fu, Marc-Alexandre Côté, Yi Tay, Chris Pal, Adam Trischler

COM-MRC: A COntext-Masked Machine Reading Comprehension Framework for Aspect Sentiment Triplet Extraction (2022.emnlp-main)

Challenge:	Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA).
Approach:	They propose a method that reframes existing machine reading comprehension (MRC) datasets as interactive, partially observable environments.
Outcome:	The proposed method "occludes" the majority of a document’s text and adds context-sensitive commands that reveal "glimpses" of the hidden text to a model.

Copied to clipboard

Zepeng Zhai, Hao Chen, Fangxiang Feng, Ruifan Li, Xiaojie Wang

Machine Reading Comprehension as Data Augmentation: A Case Study on Implicit Event Argument Extraction (2021.emnlp-main)

Challenge:	Aspect Sentiment Triplet Extraction (ASTE) aims to extract sentiment triplets from sentences, but when faced with multiple aspect terms, the MRC-based methods could fail due to the interference from other aspect terms.
Approach:	They propose a COntext-Masked MRC framework for Aspect Sentiment Triplet Extraction (ASTE) which aims to extract sentiment triplets from sentences .
Outcome:	The proposed framework outperforms state-of-the-art methods on benchmark datasets and shows that it can extract sentiment triplets from multiple aspect terms.

Copied to clipboard

Jian Liu, Yufeng Chen, Jinan Xu

Aspect-based Sentiment Analysis as Machine Reading Comprehension (2022.coling-1)

Challenge:	Existing datasets are too small to train a model for capturing regularities underlying how event arguments are extracted.
Approach:	They propose to bridge implicit EAE with machine reading comprehension (MRC) by building a unified training framework and explicit data augmentation regimes via MRC.
Outcome:	The proposed method obtains state-of-the-art performance on two benchmarks and demonstrates superior results in a data-low scenario.

Copied to clipboard

Yifei Yang, Hai Zhao

StuBot: Learning by Teaching a Conversational Agent Through Machine Reading Comprehension (2022.findings-emnlp)

Challenge:	Existing approaches to aspect-based sentiment analysis stack multiple modules and result in severe error propagation.
Approach:	They propose a MRC-PrOmpt mOdeL framework where multiple sentiment aspects are elicited by a machine reading comprehension model and their corresponding sentiment polarities are classified in a prompt learning way.
Outcome:	The proposed framework significantly outperforms existing state-of-the-art models or achieves comparable performance on widely-used benchmark datasets.

Copied to clipboard

Nayoung Jin, Hana Lee

CodeQA: A Question Answering Dataset for Source Code Comprehension (2021.findings-emnlp)

Challenge:	StuBot provides adaptive feedback for learning by teaching .
Approach:	They propose a text-based conversational agent that provides adaptive feedback for learning by teaching.
Outcome:	The proposed agent improves learning performance, immersion, and overall experience by providing adaptive feedback to the users who input the explanation text.

Copied to clipboard

Chenxiao Liu, Xiaojun Wan

Answer Span Correction in Machine Reading Comprehension (2020.findings-emnlp)

Challenge:	False. a free-form question answering dataset can serve as a useful research benchmark for source code comprehension.
Approach:	They propose a free-form question answering dataset for source code comprehension . they implement syntactic rules and semantic analysis to transform code comments into question-answer pairs.
Outcome:	The proposed dataset can serve as a useful research benchmark for source code comprehension.

Copied to clipboard

Revanth Gangi Reddy, Md Arafat Sultan, Efsun Sarioglu Kayi, Rong Zhang, Vittorio Castelli, Avi Sil

A Vietnamese Dataset for Evaluating Machine Reading Comprehension (2020.coling-main)

Challenge:	Existing MRC systems produce only partially correct answers when presented with answerable questions.
Approach:	They propose a method that yields statistically significant performance improvements over existing MRC systems.
Outcome:	The proposed method yields statistically significant performance improvements over state-of-the-art MRC systems in monolingual and multilingual evaluation.

Copied to clipboard

Kiet Nguyen, Vu Nguyen, Anh Nguyen, Ngan Nguyen

Bi-directional CognitiveThinking Network for Machine Reading Comprehension (2020.coling-main)

Challenge:	despite the lack of benchmark datasets for Vietnamese, there are few studies on machine reading comprehension (MRC) . MRC is an essential core for a range of natural language processing applications such as search engines and intelligent agents.
Approach:	They propose to use Vietnamese Question Answering Dataset to evaluate machine reading comprehension in Vietnamese . they use over 23,000 human-generated question-answer pairs based on 5,109 Vietnamese articles .
Outcome:	The proposed dataset includes over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia.

Copied to clipboard

Wei Peng, Yue Hu, Luxi Xing, Yuqiang Xie, Jing Yu, Yajing Sun, Xiangpeng Wei

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension (D18-1)

Challenge:	Existing methods for reading comprehension are still in their infancy at the level of cognitive intelligence.
Approach:	They propose a bi-directional cognitive knowledge framework to simulate reverse thinking and inertial thinking in the brain to answer questions.
Outcome:	The proposed framework shows that bi-directional knowledge helps the QA task.

Copied to clipboard

Jiahua Liu, Wan Wei, Maosong Sun, Hao Chen, Yantao Du, Dekang Lin

Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure (2020.coling-main)

Challenge:	Existing models of machine reading comprehension (MRC) are based on cloze style questions or crowdworkers given a short passage from well-edited sources.
Approach:	They propose a multi-answer multi-task framework that uses multiple reference answers for multiple questions.
Outcome:	The proposed model increases the ROUGE-L score on the DuReader dataset from 44.18, the previous state-of-the-art, to 51.09 .

Copied to clipboard

Jiaqi Li, Ming Liu, Min-Yen Kan, Zihao Zheng, Zekun Wang, Wenqiang Lei, Ting Liu, Bing Qin

BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (N19-1)

Challenge:	Multiparty dialog applications such as discourse parsing and meeting summarization are now mainstream research.
Approach:	They propose to annotate a machine reading comprehension dataset with discourse structure built over multiparty dialog using a modified Segmented Discourse Representation Theory (SDRT) style.
Outcome:	The proposed dataset contributes large-scale discourse dependency annotations in a modified Segmented Discourse Representation Theory (SDRT) style for all of its multiparty dialogs, and achieves only 67.7% F1 on Molweni’s questions, a 20+% significant drop as compared against its SQuAD 2.0 performance.

Copied to clipboard

Hu Xu, Bing Liu, Lei Shu, Philip Yu

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation (2020.coling-main)

Challenge:	Existing work on question-answering has limited training examples for RRC . question-announced questions are a key component of online commerce .
Approach:	They propose to turn customer reviews into a large source of knowledge that can be exploited to answer user questions.
Outcome:	The proposed approach improves review reading comprehension on popular language model BERT . it also improves aspect extraction and aspect sentiment classification tasks .

Copied to clipboard

Junhao Liu, Linjun Shou, Jian Pei, Ming Gong, Min Yang, Daxin Jiang

Multilinguality

Challenge:	Cross-lingual Machine Reading Comprehension (CLMRC) is a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese.
Approach:	They propose a novel approach to augment cross-lingual machine reading comprehension by combining knowledge from multiple language branch models into a single model for all target languages.
Outcome:	Extensive experiments on two CLMRC benchmarks show the proposed method is effective and robust to data noises.

Scene Restoring for Narrative Machine Reading Comprehension (2020.emnlp-main)

Copied to clipboard

Zhixing Tian, Yuanzhe Zhang, Kang Liu, Jun Zhao, Yantao Jia, Zhicheng Sheng

Multimodality

Challenge:	Narrative passages describe a chain of events, which helps the machine understand the passage comprehensively.
Approach:	They propose a method to let machine read narrative passages with their prior knowledge . they build a scene graph using Atomic as external knowledge and encode it with GDIN .
Outcome:	The proposed method achieves state-of-the-art on a Story Cloze Test and CosmosQA datasets.

Robust Machine Reading Comprehension by Learning Soft labels (2020.coling-main)

Copied to clipboard

Zhenyu Zhao, Shuangzhi Wu, Muyun Yang, Kehai Chen, Tiejun Zhao

BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels (D19-1)

Challenge:	Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels.
Approach:	They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels.
Outcome:	The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF.

Copied to clipboard

Yimin Jing, Deyi Xiong, Zhen Yan

Multilinguality

Challenge:	Using BiPaR, we build monolingual, multilingual and cross-lingual MRC on novels.
Approach:	They propose a bilingual parallel novel-style machine reading comprehension dataset BiPaR . they collect 3,667 bilingual parallel paragraphs from Chinese and English novels .
Outcome:	The proposed dataset supports multilingual and cross-lingual reading comprehension.

NumNet: Machine Reading Comprehension with Numerical Reasoning (D19-1)

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu

Interactive Language Learning by Question Answering (D19-1)

Challenge:	Existing numerical MRC models are weak in numerical reasoning, such as addition, subtraction, sorting and counting.
Approach:	They propose a numerical MRC model that integrates numerical reasoning into existing MRC models and achieves an EM-score of 64.56% on the DROP dataset.
Outcome:	The proposed model outperforms all existing machine reading comprehension models by considering the numerical relations among numbers on the DROP dataset.

Copied to clipboard

Xingdi Yuan, Marc-Alexandre Côté, Jie Fu, Zhouhan Lin, Chris Pal, Yoshua Bengio, Adam Trischler

TabComp: A Dataset for Visual Table Reading Comprehension (2025.findings-naacl)

Challenge:	Existing machine reading comprehension tasks lack interactive information-seeking component of comprehension.
Approach:	They propose a question-asking task that asks questions in a text-based environment . they propose QAit, which uses a game generator to build models that include deep reinforcement learning agents.
Outcome:	The proposed task poses questions about existence, location, and attributes of objects found in environment.

Copied to clipboard

Somraj Gautam, Abhishek Bhandari, Gaurav Harit

Multimodality

Challenge:	Existing OCR-free models struggle with complex table layouts and formatting.
Approach:	They propose a dataset to evaluate OCR-free models in table reading comprehension using table images, questions, and generative answers.
Outcome:	The proposed model outperforms existing models in the visual table reading comprehension dataset and encourages the research community to work on improving them.

MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension (2023.findings-emnlp)

Copied to clipboard

Guoxin Chen, Yiming Qian, Bowen Wang, Liangzhi Li

How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension (2023.findings-acl)

Challenge:	Existing soft prompt methods focus on designing the input-independent prompts that steer the model to fit the domain of the new dataset.
Approach:	They propose a multi-level prompt tuning method that utilizes prompts at task-specific, domain-specific and context-specific levels to enhance the comprehension of input semantics.
Outcome:	The proposed method improves on 12 benchmarks on various QA formats and achieves an average improvement of 1.94% over the state-of-the-art methods.

Copied to clipboard

Chen Zhang, Jiuheng Lin, Xiao Liu, Yuxuan Lai, Yansong Feng, Dongyan Zhao

A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)

Challenge:	Despite recent progress in multi-answer MRC, there is no systematic analysis of how this phenomenon arises and how to better address it.
Approach:	They develop a taxonomy to categorize commonly-seen multi-answer MRC instances and examine how well different paradigms deal with different types of multi-announced questions.
Outcome:	The proposed taxonomy categorizes commonly-seen multi-answer instances and analyzes how well different paradigms deal with different types of multi-announced instances.

Copied to clipboard

Yilin Niu, Fangkai Jiao, Mantong Zhou, Ting Yao, Jingfang Xu, Minlie Huang

CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)

Challenge:	Existing models for machine reading comprehension lack evidence labels for training models.
Approach:	They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process.
Outcome:	The proposed method improves on three MRC tasks on seven datasets.

Copied to clipboard

Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan

Challenge:	Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages .
Approach:	They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models .
Outcome:	The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English .

BioRead: A New Dataset for Biomedical Reading Comprehension (L18-1)

Copied to clipboard

Dimitris Pappas, Ion Androutsopoulos, Haris Papageorgiou

Coreference Reasoning in Machine Reading Comprehension (2021.acl-long)

Challenge:	BioRead is a publicly available cloze-style biomedical machine reading comprehension (MRC) dataset with 16.4 million passage-question instances.
Approach:	They propose to build a cloze-style biomedical machine reading comprehension (MRC) dataset with 16.4 million passage-question instances.
Outcome:	The proposed method outperforms baselines on bioReadLite and bioASQ, and is currently the best on BioReadLite.

Copied to clipboard

Mingzhu Wu, Nafise Sadat Moosavi, Dan Roth, Iryna Gurevych

What Makes Reading Comprehension Questions Easier? (D18-1)

Challenge:	Existing datasets for machine reading comprehension do not reflect the natural distribution and, consequently, the challenges of coreference reasoning.
Approach:	They propose to use existing coreference resolution datasets to train machine reading comprehension models to better reflect the natural distribution and, consequently, the challenges of coreference reasoning.
Outcome:	The proposed method improves the performance of state-of-the-art models on a set of coreference-related datasets.

Copied to clipboard

Saku Sugawara, Kentaro Inui, Satoshi Sekine, Akiko Aizawa

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)

Challenge:	Recent studies have shown that questions require a deeper understanding of language to answer beyond using superficial cues.
Approach:	They propose to use simple heuristics to split MRC datasets into easy and hard subsets and manually annotate questions from each subset with validity and reasoning skills to investigate which skills explain the difference between easy and harder questions.
Outcome:	The proposed model performs better for hard and easy questions than for easy questions.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou

Text Generation

Challenge:	Existing data augmentation techniques for natural language processing tasks are difficult to design.
Approach:	They propose a controllable rewriting based question data augmentation method for machine reading comprehension, question generation and question-answering natural language inference tasks.
Outcome:	The proposed method generates high-quality, high-quality question data samples on machine reading comprehension, question generation, and question-answering natural language inference tasks.

IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning (2023.findings-acl)

Copied to clipboard

Zihang Xu, Ziqing Yang, Yiming Cui, Shijin Wang

A Unified MRC Framework for Named Entity Recognition (2020.acl-main)

Challenge:	Existing systems for logical reasoning have surpassed the average performance of humans in many tasks like SQuAD but there is still a long way to go when it comes to logical reasoning.
Approach:	They propose an InDicator-Oriented Logic Pre-training task which logically strengthens pre-trained models with the help of 6 types of logical indicators and a logicalally rich dataset.
Outcome:	The proposed task achieves state-of-the-art on ReClor and LogiQA, the two most representative benchmarks in logical reasoning MRC.

Copied to clipboard

Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, Jiwei Li

Mirror: A Universal Framework for Various Information Extraction Tasks (2023.emnlp-main)

Challenge:	Named entity recognition is divided into nested NER and flat NER depending on whether entities are nesting.
Approach:	They propose to formulate named entity recognition task as machine reading comprehension task instead of sequence labeling problem .
Outcome:	The proposed framework achieves vast amount of performance boost over current models on nested and flat NER datasets.

Copied to clipboard

Tong Zhu, Junfei Ren, Zijian Yu, Mengsong Wu, Guoliang Zhang, Xiaoye Qu, Wenliang Chen, Zhefeng Wang, Baoxing Huai, Min Zhang

Have You Seen That Number? Investigating Extrapolation in Question Answering Models (2021.emnlp-main)

Challenge:	Recent studies often formulate IE tasks as a triplet extraction problem, but this paradigm does not support multi-span and n-ary extraction, leading to weak versatility.
Approach:	They propose a multi-span cyclic graph extraction problem and a non-autoregressive graph decoding algorithm to extract all spans in a single step.
Outcome:	The proposed model outperforms or reaches competitive performance with SOTA systems under few-shot and zero-shot settings and it is compatible with 57 datasets.

Copied to clipboard

Jeonghwan Kim, Giwon Hong, Kyung-min Kim, Junmo Kang, Sung-Hyon Myaeng

Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)

Challenge:	Numerical reasoning in machine reading comprehension (MRC) has shown drastic improvements over the past few years.
Approach:	They propose an E-digit number form that alleviates the lack of extrapolation in numerical MRC models.
Outcome:	The proposed model can't extrapolate to unseen numbers, the authors say . they also show that the model needs to treat numbers differently from regular words .

Copied to clipboard

Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum

Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian (2020.coling-main)

Challenge:	Current state-of-the-art machine readers do not support case-based reasoning .
Approach:	They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers.
Outcome:	The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM.

Copied to clipboard

Alena Fenogenova, Vladislav Mikhailov, Denis Shevelev

Enhancing Extractive Question Answering in Multiparty Dialogues with Logical Inference Memory Network (2025.coling-main)

Challenge:	MRC in other languages, including Russian, has not been well-addressed due to the lack of high-quality and large-scale datasets.
Approach:	They propose two Russian machine reading comprehension datasets that require reasoning over multiple sentences and commonsense knowledge to infer the answer.
Outcome:	The proposed datasets are more complex than the original ones for Russian . the results show that the proposed models are challenging for advanced models .

Copied to clipboard

Shu Zhou, Rui Zhao, Zhengda Zhou, Haohan Yi, Xuhui Zheng, Hao Wang

Lite Unified Modeling for Discriminative Reading Comprehension (2022.acl-long)

Challenge:	Existing models for multiparty dialogue question answering (QA) do not consider logical inference relations in multiparty dialogs, leading to suboptimal performance.
Approach:	They propose a memory network with logical inference for extractive QA in multiparty dialogues.
Outcome:	The proposed model achieves state-of-the-art on Molweni and FriendsQA benchmarks.

Copied to clipboard

Yilin Zhao, Hai Zhao, Libin Shen, Yinggong Zhao

ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations (2021.emnlp-main)

Challenge:	generative and discriminative MRCs focus on answer generation, extractive MRC on answer extraction.
Approach:	They propose a lightweight POS-Enhanced Iterative Co-Attention Network to handle diverse discriminative MRC tasks synchronously.
Outcome:	The proposed model improves on four discriminative MRC benchmarks.

Copied to clipboard

Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge (2022.acl-long)

Challenge:	Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations.
Approach:	They propose a machine reading comprehension dataset that leverages natural language queries to reason about the five most common event semantic relations.
Outcome:	The proposed dataset shows that current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match, F1 and event-based HIT@1 scores.

Copied to clipboard

Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Claire Cardie

Low-Resource Generation of Multi-hop Reasoning Questions (2020.acl-main)

Challenge:	Existing methods to improve machine reading comprehension (MRC) tasks require unstated knowledge to perform well.
Approach:	They propose to extract a new kind of structured knowledge from scripts and use it to improve machine reading comprehension (MRC) They propose a teacher-student paradigm to facilitate the transfer of knowledge in weakly-labeled MRC data.
Outcome:	The proposed method outperforms methods that use weakly-labeled data and improves a state-of-the-art baseline by 4.3% in accuracy on a Chinese multiple-choice MRC dataset C3.

Copied to clipboard

Jianxing Yu, Wei Liu, Shuang Qiu, Qinliang Su, Kai Wang, Xiaojun Quan, Jian Yin

Low-Resource NLP

Challenge:	Existing methods to generate valid and fluent questions from text are limited and insufficient for training.
Approach:	They propose to generate multi-hop reasoning questions from the raw text in a low resource circumstance by deducing over multiple relations on several sentences in the text.
Outcome:	The proposed model can be applied to the task of machine reading comprehension and achieve significant performance improvements.

Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension (2020.acl-main)

Copied to clipboard

Hongyu Gong, Yelong Shen, Dian Yu, Jianshu Chen, Dong Yu

MTR: A Dataset Fusing Inductive, Deductive, and Defeasible Reasoning (2023.findings-acl)

Challenge:	Existing approaches to machine reading comprehension (MRC) on long texts typically chunk text into equally-spaced segments without considering information from other segments.
Approach:	They propose to let a model learn to chunk in a more flexible way via reinforcement learning.
Outcome:	The proposed model extracts a text span from document and query as answer . previous models can only take a fixed-length (e.g., 512) text as input .

Copied to clipboard

Yitian Li, Jidong Tian, Caoyun Fan, Wenqing Chen, Hao He, Yaohui Jin

HRCA+: Advanced Multiple-choice Machine Reading Comprehension Method (2022.lrec-1)

Challenge:	Existing datasets for logical reasoning focus on monotonic logic and a single form of reasoning.
Approach:	They propose to use a dataset to study the human-like reasoning in machine reading comprehension.
Outcome:	The proposed dataset shows that state-of-the-art neural models perform noticeably worse than expected.

Copied to clipboard

Yuxiang Zhang, Hayato Yamana

Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering (2020.acl-main)

Challenge:	Multiple-choice question answering (MCQA) requires a model to understand natural languages and understand textual representations.
Approach:	They propose a model that uses human reading comprehension attention to increase accuracy for machine reading comprehension.
Outcome:	The proposed model outperforms state-of-the-art models on the Semeval-2018 Task 11 dataset and on the DREAM dataset.

Copied to clipboard

Ming Yan, Hao Zhang, Di Jin, Joey Tianyi Zhou

Knowing-how & Knowing-that: A New Task for Machine Comprehension of User Manuals (2023.findings-acl)

Challenge:	Existing MCQA datasets are small in size, which increases difficulty of model learning and generalization.
Approach:	They propose a multi-source meta transfer framework for low-resource multiple-choice question answering . they extend meta learning by incorporating multiple training sources to learn a generalized feature representation across domains .
Outcome:	The proposed framework is independent of backbone language models and can bridge the distribution gap between training sources and target.

Copied to clipboard

Hongru Liang, Jia Liu, Weihong Du, Dingnan Jin, Wenqiang Lei, Zujie Wen, Jiancheng Lv

Evaluation of Dataset Selection for Pre-Training and Fine-Tuning Transformer Language Models for Clinical Question Answering (2020.lrec-1)

Challenge:	Existing methods for machine reading comprehension of user manuals have trouble answering complex questions.
Approach:	They propose a knowing-how & knowing-that task that requires the model to answer factoid-style, procedure-style and inconsistent questions about user manuals.
Outcome:	The proposed model can answer factoid-style, procedure-style and inconsistent questions about user manuals.

Copied to clipboard

Sarvesh Soni, Kirk Roberts

To Test Machine Comprehension, Start by Defining Comprehension (2020.acl-main)

Challenge:	Existing work on EHR QA models that learn to answer questions from structured data has focused on analyzing questions or mapping questions to existing NLP based information extraction models.
Approach:	They conduct 48 experiments on two clinical question answering datasets . they use open-domain and domain-specific corpora to fine-tune Transformer language models .
Outcome:	The proposed models can learn to answer questions from unstructured notes with accuracies up to 90% on open-domain and domain-specific corpora.

Copied to clipboard

Jesse Dunietz, Greg Burnham, Akash Bharadwaj, Owen Rambow, Jennifer Chu-Carroll, Dave Ferrucci

Explaining Interactions Between Text Spans (2023.emnlp-main)

Challenge:	Existing approaches to machine reading comprehension do not adequately define comprehension, authors argue . authors argue that existing systems are not up to the task of narrative understanding as they define it .
Approach:	They propose a detailed definition of comprehension for short narratives . they argue existing systems are not up to the task of narrative understanding .
Outcome:	The proposed task definitions suggest existing systems are not up to the task of narrative understanding as they define it.

Copied to clipboard

Sagnik Ray Choudhury, Pepa Atanasova, Isabelle Augenstein

Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languages (2025.acl-long)

Challenge:	Existing highlight-based explanations focus on identifying individual important features or interactions only between adjacent tokens or tuples of tokens.
Approach:	They propose a multi-annotator dataset of human span interaction explanations for NLU and FC.
Outcome:	The proposed method compares human reasoning processes to those of a fine-tuned large language model.

Copied to clipboard

Wenhao Zhuang, Yuan Sun, Xiaobing Zhao

Rethinking Annotation: Can Language Learners Contribute? (2023.acl-long)

Challenge:	Large language models demonstrate cross-lingual transfer capabilities, but these capabilities often fail to extend to low-resource languages, especially those utilizing non-Latin scripts.
Approach:	They propose to combine character transliteration with Huffman coding to create a complete transliterations framework that can be extended to other low-resource languages.
Outcome:	The proposed framework reduces storage requirements and improves accuracy and accuracy across multiple downstream tasks while maintaining performance on high-resource languages.

Copied to clipboard

Haneul Yoo, Rifki Afina Putri, Changyoon Lee, Youngin Lee, So-Yeon Ahn, Dongyeop Kang, Alice Oh

Challenge:	Researchers have traditionally recruited native speakers to provide annotations for benchmark datasets, but there are languages for which recruiting native speakers is difficult.
Approach:	They recruit 36 language learners and provide two types of additional resources and perform mini-tests to measure their language proficiency.
Outcome:	The proposed method improves learners' language proficiency in terms of vocabulary and grammar.

Developing Dataset of Japanese Slot Filling Quizzes Designed for Evaluation of Machine Reading Comprehension (2020.lrec-1)

Copied to clipboard

Takuto Watarai, Masatoshi Tsuchiya

Bridging The Gap: Entailment Fused-T5 for Open-retrieval Conversational Machine Reading Comprehension (2023.acl-long)

Challenge:	a novel dataset of slot filling quizzes is developed for evaluation of machine reading comprehension . the dataset consists of quizzed characters of target books as slots .
Approach:	They develop a Japanese dataset of slot filling quizzes for machine reading comprehension . they define each quiz as a 4-tuple: context passage, query holding a slot, answer character and set of possible answer characters .
Outcome:	The proposed dataset includes a 4-tuple quiz, a context passage, q and a query . the q&a is performed on characters of target books, which play important roles in narrative texts .

Copied to clipboard

Xiao Zhang, Heyan Huang, Zewen Chi, Xian-Ling Mao

Context-faithful Prompting for Large Language Models (2023.findings-emnlp)

Challenge:	Open-retrieval conversational machine reading comprehension (OCMRC) simulates real-life conversation scenes.
Approach:	They propose a one-stage end-to-end framework to bridge the information gap between decision-making and question generation in a global understanding manner.
Outcome:	The proposed framework achieves new state-of-the-art performance on the OR-ShARC benchmark.

Copied to clipboard

Wenxuan Zhou, Sheng Zhang, Hoifung Poon, Muhao Chen

A Causal View of Entity Bias in (Large) Language Models (2023.findings-emnlp)

Challenge:	Large language models encode parametric knowledge about world facts but overly rely on it can cause incorrect predictions in context-sensitive NLP tasks.
Approach:	They propose to use opinion-based prompts and counterfactual demonstrations to improve LLM faithfulness to contexts.
Outcome:	The proposed methods improve faithfulness to contexts using opinion-based prompts and counterfactual demonstrations.

Copied to clipboard

Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, Muhao Chen

RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following (2025.findings-acl)

Challenge:	Entity bias affects pretrained (large) language models, causing them to rely on (biased) parametric knowledge to make unfaithful predictions.
Approach:	They propose a structured causal model whose parameters are easier to estimate . they propose to perturb the original entity with neighboring entities .
Outcome:	The proposed model reduces biasing information pertaining to the original entity while still preserving sufficient semantic information from similar entities.

Copied to clipboard

Junru Lu, Jiazheng Li, Guodong Shen, Lin Gui, Siyu An, Yulan He, Di Yin, Xing Sun

Refining Idioms Semantics Comprehension via Contrastive Learning and Cross-Attention (2024.lrec-main)

Challenge:	Existing role-playing datasets mostly contribute to controlling role style and knowledge boundaries, but overlook role-following in instruction-follower scenarios.
Approach:	They propose a fine-grained role-playing and instruction-following composite benchmark, named RoleMRC, which includes multi-turn dialogues between ideal roles and humans, including free chats or discussions upon given passages .
Outcome:	The proposed model improves instruction-following without compromising general role-playing and reasoning capabilities.

Copied to clipboard

Mingmin Wu, Guixin Su, Yongcheng Zhang, Zhongqiang Huang, Ying Sha

Title-based Extractive Summarization via MRC Framework (2024.lrec-main)

Challenge:	Existing methods based on deep learning struggle to grasp idiom semantics due to the figurative meanings of many idiomas deviating from their literal interpretations.
Approach:	They propose a Chinese idiom cloze test to capture comprehensive idiomatics and a semantic sense contrastive learning module to enhance the representation of idiomics.
Outcome:	The proposed model outperforms state-of-the-art models on the Chinese idiom cloze test and on other benchmark datasets.

Copied to clipboard

Hongjin Kim, Jai-Eun Kim, Harksoo Kim